Statistical Inference for Online Learning and Stochastic Approximation via Hierarchical Incremental Gradient Descent

Stochastic gradient descent (SGD) is an immensely popular approach for online learningin settings where data arrives in a stream or data sizes are very large. However, despite anever-increasing volume of work on SGD, much less is known about the statistical inferentialproperties of SGD-based predictions. Taking a fully inferential viewpoint, this paper introducesa novel procedure termed HiGrad to conduct statistical inference for online learning, withoutincurring additional computational cost compared with SGD. The HiGrad procedure begins byperforming SGD updates for a while and then splits the single thread into several threads, andthis procedure hierarchically operates in this fashion along each thread. With predictions pro-vided by multiple threads in place, a t-based confidence interval is constructed by decorrelatingpredictions using covariance structures given by the Ruppert–Polyak averaging scheme. Undercertain regularity conditions, the HiGrad confidence interval is shown to attain asymptoticallyexact coverage probability. Finally, the performance of HiGrad is evaluated through exten-sive simulation studies and a real data example. An R package higrad has been developed toimplement the method.